Skip to content

Connect RAG search to agent SSE flow#10

Closed
korearororo wants to merge 2 commits into
mainfrom
feat/rag-agent-sse-integration
Closed

Connect RAG search to agent SSE flow#10
korearororo wants to merge 2 commits into
mainfrom
feat/rag-agent-sse-integration

Conversation

@korearororo

@korearororo korearororo commented May 30, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Wire CORPUS_JSONL into agent startup and warm up the in-memory RAG client.
  • Implement ragSearch with fallback search when strict filters return no results.
  • Stream RAG and intent progress through POST /api/generate using SSE.
  • Preserve RAG references for downstream intent/generation handoff.
  • Add RAG reference details to the problem generation prompt and preserve source_refs.
  • Show RAG reference summaries in the web verify screen.
  • Add workflow tests for RAG fallback and RAG-to-Intent SSE events.

Verification

pnpm -F @openmath/agent typecheck
pnpm -F @openmath/agent test
pnpm -F @openmath/agent build
pnpm -F @openmath/web typecheck

Manual check:

CORPUS_JSONL=/Users/tw81512/dev/AI_HUB_data/rag_problem_generation_dataset/openmath_rag_records.jsonl pnpm -F @openmath/agent dev
curl -N -X POST http://localhost:3000/api/generate \
  -H 'Content-Type: application/json' \
  -H 'Accept: text/event-stream' \
  --data '{"grade":2,"topic_code":"9수04-04","topic_name":"도형의 닮음","mode":"structural","count":5,"difficulty":"medium","problem_type":"objective"}'

Observed:

  • RAG step starts and completes.
  • RAG references are included in the SSE payload.
  • Intent step starts and completes.
  • Workflow then returns PIPELINE_NOT_IMPLEMENTED because generation/verification steps are outside this RAG integration change.

Notes

  • This PR does not implement the full generation/verification pipeline.
  • The RAG JSONL file is intentionally not committed to the repository.
  • Runtime requires CORPUS_JSONL to point to openmath_rag_records.jsonl.

Summary by cubic

Connects RAG retrieval to the agent’s SSE pipeline and completes the end-to-end verification flow. Streams RAG → Intent → Generate → SymPy → Re-solve → Objective Map via POST /api/generate, with source references and final results rendered in @openmath/web.

  • New Features

    • Full verification workflow with SSE: step indices, summaries, and a final result event with per-candidate outcomes.
    • RAG: load corpus from CORPUS_JSONL, warm in-memory client, smart fallback, and preserve source_refs.
    • LLM agents: generator, constraint critic, refiner, and independent solver wired via openai-compatible or openai using a filesystem prompt loader; safe fallbacks when LLM config is missing.
    • Math engine: client for /solve and /verify; SymPy verification with equation extraction/normalization; server now solves linear systems.
    • Objective mapping gate validates target code and preserved dimensions.
    • Strategies: add 12 achievement-standard YAMLs plus schema loader and tests.
    • Web: verify screen shows top RAG references; persist last streamed results for the results view; better LaTeX normalization.
    • Tests: workflow covers RAG fallback and the full SSE sequence; loader tests ensure strategy coverage.
  • Migration

    • Set CORPUS_JSONL to the path of openmath_rag_records.jsonl before starting @openmath/agent.
    • Ensure MATH_ENGINE_URL points to a running math engine; without it, verification cannot complete.
    • Optionally configure LLM: LLM_PROVIDER=openai-compatible, LLM_BASE_URL, LLM_API_KEY, LLM_MODEL (or LLM_PROVIDER=openai + LLM_API_KEY). Fallback generation is used if not set.

Written for commit f0f71bd. Summary will update on new commits.

Review in cubic

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

7 issues found across 14 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/agent/src/steps/intent-extraction.ts">

<violation number="1" location="packages/agent/src/steps/intent-extraction.ts:90">
P2: Objective-code validation should trim candidate strings before regex matching; otherwise whitespace-padded valid codes are downgraded to the default fallback code.</violation>

<violation number="2" location="packages/agent/src/steps/intent-extraction.ts:98">
P2: The objective code regex is declared inline in `isObjectiveCode`, duplicating the same pattern already defined in `intent.schema.ts` and `strategy.schema.ts`. Export a shared constant (e.g., from `intent.schema.ts`) and reuse it here to prevent pattern drift.</violation>
</file>

<file name="packages/agent/src/server/sse/progress-stream.ts">

<violation number="1" location="packages/agent/src/server/sse/progress-stream.ts:11">
P2: Unhandled exceptions from the workflow generator terminate the SSE stream without sending a protocol-level `error` event.</violation>

<violation number="2" location="packages/agent/src/server/sse/progress-stream.ts:27">
P2: `"info"` step status is incorrectly mapped to `"completed"`. The `StepStatusSchema` includes `"info"` as a valid status (alongside `"start"` and `"done"`), but the ternary `event.status === "start" ? "started" : "completed"` lumps `"info"` into `"completed"`. The SSE consumer will interpret an `"info"` event as step completion, which misrepresents the actual workflow state.</violation>
</file>

<file name="packages/agent/src/tools/math-engine-client.ts">

<violation number="1" location="packages/agent/src/tools/math-engine-client.ts:94">
P2: When the request times out, `controller.abort()` fires without a reason, causing `fetch` to reject with a generic `AbortError` (`DOMException: The operation was aborted`). The caller gets no indication of which endpoint or timeout duration caused the failure, making runtime debugging harder.</violation>
</file>

<file name="packages/agent/src/workflows/verification-workflow.ts">

<violation number="1" location="packages/agent/src/workflows/verification-workflow.ts:63">
P1: Missing error boundary: if `ragSearch()` or `extractIntent()` throws, the generator crashes without yielding an SSE error event, leaving the client mid-stream with no graceful error signal.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread packages/agent/src/steps/rag-search.ts Outdated
};

const ragStarted = Date.now();
const rag = await ragSearch({ rag: deps.rag }, { request });

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: Missing error boundary: if ragSearch() or extractIntent() throws, the generator crashes without yielding an SSE error event, leaving the client mid-stream with no graceful error signal.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/agent/src/workflows/verification-workflow.ts, line 63:

<comment>Missing error boundary: if `ragSearch()` or `extractIntent()` throws, the generator crashes without yielding an SSE error event, leaving the client mid-stream with no graceful error signal.</comment>

<file context>
@@ -43,10 +45,84 @@ export type WorkflowReturn = {
+  };
+
+  const ragStarted = Date.now();
+  const rag = await ragSearch({ rag: deps.rag }, { request });
+
+  yield {
</file context>


function firstValidObjectiveCode(candidates: Array<string | null | undefined>): string {
for (const candidate of candidates) {
if (candidate !== undefined && candidate !== null && isObjectiveCode(candidate)) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Objective-code validation should trim candidate strings before regex matching; otherwise whitespace-padded valid codes are downgraded to the default fallback code.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/agent/src/steps/intent-extraction.ts, line 90:

<comment>Objective-code validation should trim candidate strings before regex matching; otherwise whitespace-padded valid codes are downgraded to the default fallback code.</comment>

<file context>
@@ -27,7 +28,73 @@ export interface IntentExtractionOutput {
+
+function firstValidObjectiveCode(candidates: Array<string | null | undefined>): string {
+  for (const candidate of candidates) {
+    if (candidate !== undefined && candidate !== null && isObjectiveCode(candidate)) {
+      return candidate;
+    }
</file context>

Comment on lines +11 to +13
for await (const event of events) {
await stream.writeSSE(toWireEvent(event));
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Unhandled exceptions from the workflow generator terminate the SSE stream without sending a protocol-level error event.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/agent/src/server/sse/progress-stream.ts, line 11:

<comment>Unhandled exceptions from the workflow generator terminate the SSE stream without sending a protocol-level `error` event.</comment>

<file context>
@@ -2,11 +2,88 @@
+  events: AsyncGenerator<ProgressEvent, unknown, void>,
 ): Promise<void> {
-  throw new Error("pipeProgressToSse: not implemented yet");
+  for await (const event of events) {
+    await stream.writeSSE(toWireEvent(event));
+  }
</file context>
Suggested change
for await (const event of events) {
await stream.writeSSE(toWireEvent(event));
}
try {
for await (const event of events) {
await stream.writeSSE(toWireEvent(event));
}
} catch (error) {
await stream.writeSSE({
event: "error",
data: JSON.stringify({
stage: "orchestrator",
message: error instanceof Error ? error.message : String(error),
code: "WORKFLOW_STREAM_FAILED",
recoverable: false,
}),
});
}

timeoutMs: number,
): Promise<T> {
const controller = new AbortController();
const timeout = setTimeout(() => controller.abort(), timeoutMs);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: When the request times out, controller.abort() fires without a reason, causing fetch to reject with a generic AbortError (DOMException: The operation was aborted). The caller gets no indication of which endpoint or timeout duration caused the failure, making runtime debugging harder.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/agent/src/tools/math-engine-client.ts, line 94:

<comment>When the request times out, `controller.abort()` fires without a reason, causing `fetch` to reject with a generic `AbortError` (`DOMException: The operation was aborted`). The caller gets no indication of which endpoint or timeout duration caused the failure, making runtime debugging harder.</comment>

<file context>
@@ -68,7 +68,45 @@ export interface MathEngineClientOptions {
+  timeoutMs: number,
+): Promise<T> {
+  const controller = new AbortController();
+  const timeout = setTimeout(() => controller.abort(), timeoutMs);
+
+  try {
</file context>
Suggested change
const timeout = setTimeout(() => controller.abort(), timeoutMs);
const timeout = setTimeout(() => controller.abort(new Error(`math-engine ${path} timed out after ${timeoutMs}ms`)), timeoutMs);

return "9수00-00";
}

function isObjectiveCode(value: string): boolean {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: The objective code regex is declared inline in isObjectiveCode, duplicating the same pattern already defined in intent.schema.ts and strategy.schema.ts. Export a shared constant (e.g., from intent.schema.ts) and reuse it here to prevent pattern drift.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/agent/src/steps/intent-extraction.ts, line 98:

<comment>The objective code regex is declared inline in `isObjectiveCode`, duplicating the same pattern already defined in `intent.schema.ts` and `strategy.schema.ts`. Export a shared constant (e.g., from `intent.schema.ts`) and reuse it here to prevent pattern drift.</comment>

<file context>
@@ -27,7 +28,73 @@ export interface IntentExtractionOutput {
+  return "9수00-00";
+}
+
+function isObjectiveCode(value: string): boolean {
+  return /^(9수|10공수)\d{2}-\d{2}$/.test(value);
 }
</file context>

data: JSON.stringify({
index: stepIndex(event.step),
name: event.step,
status: event.status === "start" ? "started" : "completed",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: "info" step status is incorrectly mapped to "completed". The StepStatusSchema includes "info" as a valid status (alongside "start" and "done"), but the ternary event.status === "start" ? "started" : "completed" lumps "info" into "completed". The SSE consumer will interpret an "info" event as step completion, which misrepresents the actual workflow state.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/agent/src/server/sse/progress-stream.ts, line 27:

<comment>`"info"` step status is incorrectly mapped to `"completed"`. The `StepStatusSchema` includes `"info"` as a valid status (alongside `"start"` and `"done"`), but the ternary `event.status === "start" ? "started" : "completed"` lumps `"info"` into `"completed"`. The SSE consumer will interpret an `"info"` event as step completion, which misrepresents the actual workflow state.</comment>

<file context>
@@ -2,11 +2,88 @@
+        data: JSON.stringify({
+          index: stepIndex(event.step),
+          name: event.step,
+          status: event.status === "start" ? "started" : "completed",
+          summary: stepSummary(event),
+          raw: event,
</file context>
Suggested change
status: event.status === "start" ? "started" : "completed",
status: event.status === "start" ? "started" : event.status === "info" ? "info" : "completed",

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

14 issues found across 46 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/agent/src/steps/intent-extraction.ts">

<violation number="1" location="packages/agent/src/steps/intent-extraction.ts:36">
P2: LLM path in `extractIntent` has no error handling, so failures during `generateObject` (network timeout, API error, rate limit, malformed response) propagate as unhandled exceptions instead of falling back to `fallbackExtractIntent`. The deterministic fallback implementation exists and is available, but the code only reaches it when `deps.model` or `deps.prompts` is undefined — not when the LLM call itself fails.</violation>
</file>

<file name="packages/web/app/app/new/verify/view.tsx">

<violation number="1" location="packages/web/app/app/new/verify/view.tsx:172">
P1: useEffect called after early return — violates React Rules of Hooks. If `valid` toggles between renders, React will detect a hook count mismatch, causing an error or state corruption.</violation>
</file>

<file name="packages/web/app/app/new/result/view.tsx">

<violation number="1" location="packages/web/app/app/new/result/view.tsx:89">
P2: Using an unscoped `openmath:last-results` cache can show stale problems for unrelated grade/topic/mode selections.</violation>

<violation number="2" location="packages/web/app/app/new/result/view.tsx:398">
P2: `missingDims` is unconditionally set to `[]`, which silently suppresses the dimension-warning notice for non-pass live results</violation>
</file>

<file name="packages/agent/src/config/env.ts">

<violation number="1" location="packages/agent/src/config/env.ts:55">
P2: Key not trimmed after splitting on `=`. Lines like `KEY = value` (with spaces around `=`) produce `key = "KEY "` (trailing space), causing `process.env["KEY "]` to be set instead of `process.env["KEY"]`. The existing env-var check `process.env[key] !== undefined` then fails to match the intended variable, silently breaking configuration loading for anyone using spaced assignment — the most common non-standard .env format variants.</violation>
</file>

<file name="packages/web/components/math/latex-renderer.tsx">

<violation number="1" location="packages/web/components/math/latex-renderer.tsx:63">
P2: The `sqrt` regex `[^()]+` cannot handle nested parentheses (e.g. `sqrt((a+b))`, `sqrt(sin(x))`), causing silently incorrect KaTeX rendering instead of a square root symbol.</violation>

<violation number="2" location="packages/web/components/math/latex-renderer.tsx:64">
P1: The `**` exponent regex greedily captures only the first safe token into the exponent group. Inputs like `x ** (y+1)` produce garbled LaTeX (`x^{(y}+1)` instead of `x^{y+1}` or `x^{(y+1)}`).</violation>
</file>

<file name="packages/agent/src/tools/prompt-loader.ts">

<violation number="1" location="packages/agent/src/tools/prompt-loader.ts:88">
P2: Path traversal risk in `readPrompt`: the `id` parameter is interpolated directly into `resolve(promptsDir, \`${id}.md\`)` without verifying the resolved path stays inside `promptsDir`. An `id` like `../../etc/somefile` could read arbitrary `.md` files anywhere on the filesystem. This is a defense-in-depth concern even though current callers pass controlled IDs.</violation>
</file>

<file name="packages/agent/src/steps/sympy-verification.ts">

<violation number="1" location="packages/agent/src/steps/sympy-verification.ts:85">
P1: `extractEquation` regex captures natural-language text as part of the equation because `A-Za-z` in the character class allows English words. E.g., "What is the solution to 2x+3=7?" extracts "What is the solution to 2x+3=7" instead of just "2x+3=7", causing a misleading `math_engine_error` rather than `equation_not_found`.</violation>

<violation number="2" location="packages/agent/src/steps/sympy-verification.ts:89">
P2: `normalizeMathText` does not replace `\cdot` with `*`. Since `\` is not in the `extractEquation` character class, equations containing `\cdot` produce garbled extractions like `"cdot b = c"` instead of `"a * b = c"`, leading to spurious math engine errors.</violation>
</file>

<file name="packages/agent/.env.example">

<violation number="1" location="packages/agent/.env.example:6">
P2: The provider example now uses `openai-compatible`, but the documented provider options in the same file still mention `cliproxy` (an invalid value). This can cause `.env` setup failures.</violation>
</file>

<file name="packages/agent/data/README.md">

<violation number="1" location="packages/agent/data/README.md:34">
P3: Avoid committing a developer-specific absolute corpus path in docs; use a portable placeholder path to prevent misleading setup instructions.</violation>
</file>

<file name="packages/agent/src/agents/generator-agent.ts">

<violation number="1" location="packages/agent/src/agents/generator-agent.ts:58">
P1: `inferred_intent` is sourced from model output (except objective_code), which can make objective mapping fail due to intent-dimension drift. Use the requested intent as the canonical value here.</violation>
</file>

<file name="packages/agent/src/tools/schema-loader.ts">

<violation number="1" location="packages/agent/src/tools/schema-loader.ts:77">
P1: Validate or sanitize `code` before resolving the strategy file path to prevent directory traversal outside `strategiesDir`.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

const isDone = stream.status === "done";
const showPreview = stream.previewLatex !== null;

useEffect(() => {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: useEffect called after early return — violates React Rules of Hooks. If valid toggles between renders, React will detect a hook count mismatch, causing an error or state corruption.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/web/app/app/new/verify/view.tsx, line 172:

<comment>useEffect called after early return — violates React Rules of Hooks. If `valid` toggles between renders, React will detect a hook count mismatch, causing an error or state corruption.</comment>

<file context>
@@ -169,6 +169,15 @@ export function VerifyView({ grade, topic, mode, dims }: Props) {
   const isDone = stream.status === "done";
   const showPreview = stream.previewLatex !== null;
 
+  useEffect(() => {
+    if (isDone && stream.candidates.length > 0) {
+      window.localStorage.setItem(
</file context>

function normalizeMathNotation(input: string): string {
return input
.replace(/\bsqrt\(([^()]+)\)/g, "\\sqrt{$1}")
.replace(/([A-Za-z0-9)\]}]+)\s*\*\*\s*([A-Za-z0-9({[]+)/g, "$1^{$2}")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: The ** exponent regex greedily captures only the first safe token into the exponent group. Inputs like x ** (y+1) produce garbled LaTeX (x^{(y}+1) instead of x^{y+1} or x^{(y+1)}).

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/web/components/math/latex-renderer.tsx, line 64:

<comment>The `**` exponent regex greedily captures only the first safe token into the exponent group. Inputs like `x ** (y+1)` produce garbled LaTeX (`x^{(y}+1)` instead of `x^{y+1}` or `x^{(y+1)}`).</comment>

<file context>
@@ -56,6 +58,14 @@ function renderLatex(latex: string, block: boolean): string {
+function normalizeMathNotation(input: string): string {
+  return input
+    .replace(/\bsqrt\(([^()]+)\)/g, "\\sqrt{$1}")
+    .replace(/([A-Za-z0-9)\]}]+)\s*\*\*\s*([A-Za-z0-9({[]+)/g, "$1^{$2}")
+    .replace(/([0-9A-Za-z)\]}]+)\s*\*\s*([A-Za-z({[]+)/g, "$1$2")
+    .replace(/\^\{([^{}]+)\}\s*\)/g, "^{$1})");
</file context>


export function extractEquation(text: string): string | null {
const normalized = normalizeMathText(text);
const match = normalized.match(/[A-Za-z0-9+\-*/^().\s]+=[A-Za-z0-9+\-*/^().\s]+/);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: extractEquation regex captures natural-language text as part of the equation because A-Za-z in the character class allows English words. E.g., "What is the solution to 2x+3=7?" extracts "What is the solution to 2x+3=7" instead of just "2x+3=7", causing a misleading math_engine_error rather than equation_not_found.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/agent/src/steps/sympy-verification.ts, line 85:

<comment>`extractEquation` regex captures natural-language text as part of the equation because `A-Za-z` in the character class allows English words. E.g., "What is the solution to 2x+3=7?" extracts "What is the solution to 2x+3=7" instead of just "2x+3=7", causing a misleading `math_engine_error` rather than `equation_not_found`.</comment>

<file context>
@@ -16,8 +16,87 @@ export interface SympyVerificationOutput {
+
+export function extractEquation(text: string): string | null {
+  const normalized = normalizeMathText(text);
+  const match = normalized.match(/[A-Za-z0-9+\-*/^().\s]+=[A-Za-z0-9+\-*/^().\s]+/);
+  return match?.[0]?.trim() ?? null;
+}
</file context>

Comment on lines +58 to +61
inferred_intent: {
...result.object.inferred_intent,
objective_code: input.intent.objective_code,
},

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: inferred_intent is sourced from model output (except objective_code), which can make objective mapping fail due to intent-dimension drift. Use the requested intent as the canonical value here.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/agent/src/agents/generator-agent.ts, line 58:

<comment>`inferred_intent` is sourced from model output (except objective_code), which can make objective mapping fail due to intent-dimension drift. Use the requested intent as the canonical value here.</comment>

<file context>
@@ -25,9 +30,45 @@ export interface GeneratorAgent {
+        source_refs: result.object.source_refs.length
+          ? result.object.source_refs
+          : input.refs.map((ref) => ref.item_id),
+        inferred_intent: {
+          ...result.object.inferred_intent,
+          objective_code: input.intent.objective_code,
</file context>
Suggested change
inferred_intent: {
...result.object.inferred_intent,
objective_code: input.intent.objective_code,
},
inferred_intent: input.intent,

strategiesDir: string,
code: string,
): Promise<Strategy | null> {
const path = resolve(strategiesDir, `${code}.yaml`);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: Validate or sanitize code before resolving the strategy file path to prevent directory traversal outside strategiesDir.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/agent/src/tools/schema-loader.ts, line 77:

<comment>Validate or sanitize `code` before resolving the strategy file path to prevent directory traversal outside `strategiesDir`.</comment>

<file context>
@@ -22,7 +31,73 @@ export interface FsStrategyLoaderOptions {
+  strategiesDir: string,
+  code: string,
+): Promise<Strategy | null> {
+  const path = resolve(strategiesDir, `${code}.yaml`);
+  let file: string;
+  try {
</file context>
Suggested change
const path = resolve(strategiesDir, `${code}.yaml`);
const safeCode = basename(code);
if (safeCode !== code) {
throw new Error(`Invalid strategy code: ${code}`);
}
const path = resolve(strategiesDir, `${safeCode}.yaml`);

}

async function readPrompt(promptsDir: string, id: string): Promise<LoadedPrompt> {
const path = resolve(promptsDir, `${id}.md`);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Path traversal risk in readPrompt: the id parameter is interpolated directly into resolve(promptsDir, \${id}.md`)without verifying the resolved path stays insidepromptsDir. An idlike../../etc/somefilecould read arbitrary.md` files anywhere on the filesystem. This is a defense-in-depth concern even though current callers pass controlled IDs.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/agent/src/tools/prompt-loader.ts, line 88:

<comment>Path traversal risk in `readPrompt`: the `id` parameter is interpolated directly into `resolve(promptsDir, \`${id}.md\`)` without verifying the resolved path stays inside `promptsDir`. An `id` like `../../etc/somefile` could read arbitrary `.md` files anywhere on the filesystem. This is a defense-in-depth concern even though current callers pass controlled IDs.</comment>

<file context>
@@ -37,7 +58,51 @@ export interface FsPromptLoaderOptions {
+}
+
+async function readPrompt(promptsDir: string, id: string): Promise<LoadedPrompt> {
+  const path = resolve(promptsDir, `${id}.md`);
+  const file = await readFile(path, "utf8");
+  const parsed = matter(file);
</file context>

const [liveProblems, setLiveProblems] = useState<ResultProblem[] | null>(null);

useEffect(() => {
const raw = window.localStorage.getItem("openmath:last-results");

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Using an unscoped openmath:last-results cache can show stale problems for unrelated grade/topic/mode selections.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/web/app/app/new/result/view.tsx, line 89:

<comment>Using an unscoped `openmath:last-results` cache can show stale problems for unrelated grade/topic/mode selections.</comment>

<file context>
@@ -83,10 +83,27 @@ export function ResultView({
+  const [liveProblems, setLiveProblems] = useState<ResultProblem[] | null>(null);
+
+  useEffect(() => {
+    const raw = window.localStorage.getItem("openmath:last-results");
+    if (raw === null) return;
+    try {
</file context>

@@ -16,8 +16,87 @@ export interface SympyVerificationOutput {
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: normalizeMathText does not replace \cdot with *. Since \ is not in the extractEquation character class, equations containing \cdot produce garbled extractions like "cdot b = c" instead of "a * b = c", leading to spurious math engine errors.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/agent/src/steps/sympy-verification.ts, line 89:

<comment>`normalizeMathText` does not replace `\cdot` with `*`. Since `\` is not in the `extractEquation` character class, equations containing `\cdot` produce garbled extractions like `"cdot b = c"` instead of `"a * b = c"`, leading to spurious math engine errors.</comment>

<file context>
@@ -16,8 +16,87 @@ export interface SympyVerificationOutput {
+  return match?.[0]?.trim() ?? null;
+}
+
+export function normalizeMathText(text: string): string {
+  return text
+    .replace(/\\\((.*?)\\\)/g, "$1")
</file context>


# LLM Provider: "openai" or "cliproxy"
LLM_PROVIDER=openai
LLM_PROVIDER=openai-compatible

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: The provider example now uses openai-compatible, but the documented provider options in the same file still mention cliproxy (an invalid value). This can cause .env setup failures.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/agent/.env.example, line 6:

<comment>The provider example now uses `openai-compatible`, but the documented provider options in the same file still mention `cliproxy` (an invalid value). This can cause `.env` setup failures.</comment>

<file context>
@@ -3,13 +3,12 @@ MATH_ENGINE_URL=http://localhost:8000
 
 # LLM Provider: "openai" or "cliproxy"
-LLM_PROVIDER=openai
+LLM_PROVIDER=openai-compatible
+LLM_BASE_URL=http://localhost:8080/v1
+LLM_API_KEY=dummy-key
</file context>

런타임 corpus는 `openmath-rag-record-v1` JSONL이다. 기본 로컬 경로:

```text
/Users/tw81512/dev/AI_HUB_data/rag_problem_generation_dataset/openmath_rag_records.jsonl

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: Avoid committing a developer-specific absolute corpus path in docs; use a portable placeholder path to prevent misleading setup instructions.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/agent/data/README.md, line 34:

<comment>Avoid committing a developer-specific absolute corpus path in docs; use a portable placeholder path to prevent misleading setup instructions.</comment>

<file context>
@@ -5,23 +5,43 @@
+런타임 corpus는 `openmath-rag-record-v1` JSONL이다. 기본 로컬 경로:
+
+```text
+/Users/tw81512/dev/AI_HUB_data/rag_problem_generation_dataset/openmath_rag_records.jsonl
+```
+
</file context>
Suggested change
/Users/tw81512/dev/AI_HUB_data/rag_problem_generation_dataset/openmath_rag_records.jsonl
/absolute/path/to/openmath_rag_records.jsonl

smilebank7 added a commit that referenced this pull request Jun 2, 2026
* Add 12 achievement-standard strategy YAMLs

Replace placeholder strategies with full demo set covering middle-school
units 9수01-01..9수04-05 (12 files total). Each YAML conforms to
StrategySchema with techniques, evaluation_dimensions, and structural
or conceptual transforms. Sourced from PR #10 (@korearororo).

* Document local RAG corpus policy

Add corpus directory placeholder so the path exists for fresh clones,
expand data/README.md with the corpus + strategy YAML conventions, and
ignore non-fixture JSONL under data/corpus so large local corpora stay
out of the repo. Sourced from PR #10 (@korearororo).

* Pin capstone demo scope

Add docs/product/DEMO_SCOPE.md describing the fixed demo flow (중3 /
이차방정식 / 구조 동형 / 3문항) and link it from the top-level README.
Sourced from PR #10 (@korearororo).

* Support linear systems in math-engine /solve

Allow SolveRequest.equation and .variable to be either a string or a
list of strings. When both are lists, parse each equation and solve the
system via SymPy's multi-variable solve, returning each solution as a
comma-joined assignment string. Single-equation behavior is preserved.
Add test_solve_linear_system covering x+y=5, x-y=1. Sourced from PR #10
(@korearororo).

* Test fs strategy loader against demo set

Verify createFsStrategyLoader loads 9수02-09 by code, exposes
must_preserve dimensions, and loadAll returns the full 12-strategy demo
set. Sourced from PR #10 (@korearororo).
@smilebank7

Copy link
Copy Markdown
Contributor

Closing in favor of #11.

PR #11 cherry-picked the non-overlapping assets from this branch onto current main (12 strategy YAMLs, corpus policy, capstone demo scope, math-engine linear systems support, strategy-loader integration test). The remaining agent/step/workflow/server/web changes here overlap with what shipped via #9 and would regress current main, so they are intentionally not brought over.

Thanks @korearororo!

@smilebank7 smilebank7 closed this Jun 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants